Upgrade documentation environments to Sphinx 9 - #2599

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mdboom I opened this draft PR for #2598. It updates the shared CI documentation environment from Sphinx <8.2.0 to Sphinx 9.x, along with the three component Pixi manifests and the local setup script. The local syntax and consistency checks pass; the full docs build is intended to run in CI because Pixi is not installed locally. Please let me know if you would like any changes, or if the team would prefer to handle this upgrade internally. Thanks!

/ok to test 53b0e5e

I pushed a follow-up fix to the PR branch. The Sphinx 9 upgrade was leaving sphinx-toolbox on the conda-forge 4.1.2 line, which is incompatible with Sphinx 9. The docs environments now install sphinx-toolbox>=4.2.0 from PyPI consistently across Pixi, the shared conda environment, and the setup script. Local TOML/YAML parsing, shell syntax, and git diff --check pass. Could you please rerun CI when convenient?

/ok to test 5e75843

Head is now 9d4fc2c6. Two changes since the last run:

1. Merged current main. The branch was 183 commits behind, which is what the red test matrix was actually reporting. Every Test * job failed at collection with:

ERROR collecting tests/test_utils.py
tests/test_utils.py::TestViewCPU::test_args_viewable_as_strided_memory_cpu: in "parametrize"
the number of names (1) must be equal to the number of values (3)

That is #2212 (tests: fix parametrize patterns rejected by pytest 9.1.0), which landed on main after this branch was cut, together with the pytest==9.1.0 pin in cuda_core/pyproject.toml. It has nothing to do with the docs environments. The one build failure, Build linux-64, CUDA 13.3.0 / py3.10, was a registry read error pulling the manylinux image:

short read: expected 71891324 bytes but got 2097153: unexpected EOF
##[error]cibuildwheel: Command ['docker', 'create', ... ] failed with code 1

The other 17 build jobs on that run passed. Because docs has needs: build-linux-64, the Docs job was skipped, so the Sphinx 9 build itself has not been exercised in CI yet. That is the result worth getting from the next run.

2. Dropped the conda sphinx-toolbox from the three Pixi manifests. My previous commit added sphinx-toolbox>=4.2.0 under [feature.docs.pypi-dependencies] but left sphinx-toolbox = "*" under [feature.docs.dependencies]. conda-forge tops out at 4.1.2, which does not support Sphinx 9, and cuda_core/pixi.lock confirms that is what currently resolves:

conda: https://conda.anaconda.org/conda-forge/noarch/sphinx-toolbox-4.1.2-pyhd8ed1ab_0.conda

With the package declared on both sides the conda build stays in the solve and the >=4.2.0 constraint never takes effect. The manifests now match environment-docs.yml, which already took it from PyPI only.

One thing I cannot do locally: pixi.lock still records sphinx <8.2.0 and sphinx-toolbox 4.1.2. I do not have Pixi installed, so I have not regenerated the lock files. ci-pixi-source-test.yml does not pass --frozen, so it should re-solve rather than fail, but let me know if you want the locks regenerated in this PR and I will find a way to run pixi lock.

Could you rerun CI when convenient? @mdboom

Follow-up: I installed Pixi and verified the solve. The result is stronger than I described above, so recording it here.

On the previous head 5e75843, the docs environment does not solve at all. Running pixi lock in cuda_core/:

Error:   × failed to solve the pypi requirements of environment 'docs' for platform 'linux-aarch64'
  ├─▶ failed to resolve pypi dependencies
  ╰─▶ Because you require sphinx-toolbox>=4.2.0 and sphinx-toolbox==4.1.2, we
      can conclude that your requirements are unsatisfiable.
  help: The following PyPI packages have been pinned by the conda solve, and
        this version may be causing a conflict:
        sphinx-toolbox==4.1.2

Pixi pins the PyPI resolution to whatever the conda solve chose, so declaring sphinx-toolbox in both [feature.docs.dependencies] and [feature.docs.pypi-dependencies] is not merely redundant, it is unsatisfiable. Dropping the conda entry is required, not a cleanup.

On the current head 9d4fc2c6 the solve succeeds and produces what the upgrade is aiming for:

sphinx 9.1.0                       (conda-forge)
sphinx-toolbox 4.3.0               (PyPI wheel)
nvidia-sphinx-theme 0.0.9.post1    (PyPI wheel)

The regenerated lock stays at version: 7, matching the format the pinned PIXI_VERSION: "v0.73.0" in ci-pixi-source-test.yml expects.

I have not committed the regenerated pixi.lock files. My local Pixi is 0.76.2 and CI pins 0.73.0, and I would rather not land a 20k-line lock diff written by a different Pixi than the one the repo standardizes on. The three pixi.lock files therefore still record sphinx <8.2.0 and sphinx-toolbox 4.1.2, which is now out of step with the manifests. Given #2532 relocked in-repo after #2384, I assume you want them regenerated. Say the word and I will install Pixi 0.73.0 and push the locks, or leave it to whoever runs the relock normally.

Docs / Build docs uses cuda_python/docs/environment-docs.yml through conda rather than Pixi, so the CI docs build is a separate path from the above and still needs a run to confirm. @mdboom

/ok to test 9d4fc2c